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ABSTBACT / 

The' report provides a summary of the independent, 
third party evaluation of the 1972-73 funding year of the Guided 
Occupational Orientation Project of the Syracuse City School / 
District. ^A systems approach to evaluation was utilized in this 
effort. The funded project involved all three grade 

populations — elementary, junior high school, and senior high school. 
Each major grade grouping had a distinct focus: (1) 
elementary — career awareness, (2) junior high school--career 
orientation and exploration, and (3) senior high, school — job training 
and placement. Each group goal was supporiSed by separate grade level 
goals. The variables for each target group — students, parents, and 
industry-are described in terms of the actual effect of the program 
to date. In Assessing the overall impact of the project, the major 
findings were: (1) career awareness is increasing generally for / 
students in grades five through* nine; (2) attitudes are not being 
affected, therefore, attitudinal concepts would be discarded; and .(3) 



in grades 10 



-12, the occupational learning centers are having the 



desired impact on the target pdpulation. (VA) 
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Chapter I • 
..--^ Introduction 

This report provides a summary o£ the independent, third-party 
evaluation conducted by Educational Services, Incorporated (ESI) 
for the 1972-73^ funding y^ar (ending January 31, 1973) of the 
Guided Occupational Orientation Project (GOOP) of the Syracuse 
^City School District (SCSD) . 

ESI evaluation^ team membqrs have approached this assignment , 
.with the conviction that evaluation^iV^nbt only how to assess the 
impact of any specific objective of educational activity, but 
rather how evaluation can be utiilized to create a robust and 
dynamic planning system. 

Evaluation often has been perceijyed as a rrecessary evil re- 
quired by a supervisory agency as a basis for continued^^^d/^r 
additional funding. Results of such evaluation often produce self- 
justification of programs; tedious summaries of quantitative^ data 
with little to no regard for qualitative data gathering or pre- 
sentation; or opinionated and emotional observations. 

Also, evaluation is often equated with research.. The research 
paradigm of evaluation views the procedure as a fact finding 
mechanism to determine the worth of programs. A central weakness 
of the application of the formalized research techniques to 
evaluatioh rests in the problem of implementation of, or useful- 
CD ' 



ness of,. such data in making meaningful change in educational 
programs. While researchers are" fascinated by such data, educators 
are unable vO use research data in administration of specific 

^i;ograms . ' , 

Therefore, ESI has employed an alternative approach to 
evaluation which .might be characterized as a systems approach, 
ESI views, evaluation as a means to facilitate and to increase the 
flexibility of- education- so that teachers and project staff admin- 
istration can participate directly in the process of evaluation. 

The evaluation model designed by ESI recognizes the exten- 
siveness of a particular ' educational project; that such a project 
contains many elements and parts; that complexities of relation- 
ships, linkages, and interconnections of parts occur often and 
rapidly; and that the dynamic nature of educational projects is 
subject to change and vari^nces^. 

The systems approach to evaluation used by ESI is summarized 
in Figure I. The first step is the careful selection of the 
variables (behavior of students, teachers, parents, etc.) upon 
which change is to be effected. Usually ,." each identified triable 
categorizes, a major, project activity. 

E.ach major activity 'of a project is considered a separate feed 
back ;oop and a goal is established to provide directions for the 
total system in terms of desired outcomes. 

After identification of each -goal , indicators and/or data 
gatharing mechanisms which reflect upon the specified goal are 
selected. • . . 

5 



As data and/or observations are gathered and matched against - 
specific goals, the decision-37;akers o£ the project are able to 
clearly identify successes, lack of progress, or inability of the 
project to effect change on the specific variab-le . This serves' 
to pinpoint areas of strength and weakness quickly and "graphically. 
Thus, modifications can be made as necessary. The style of eval- 
uation i^ concerned both with demonstrating overall, long-term 
program results as well as assisting educational administrators 
in improving the quality of programs over tim? without waiting 

4 

.ntil this final evaluation, -when it is too late to modify. ^ , 

The ESI evaluation scheme which serves these functions is 
based on the belief that adininis trators of educational programs 
must know the following: what the program is planning to accomplish 
what will determine this accomplishment; and what techniques are 
necessary for this accomplishment. 

The five major elements which compose the evaluation scheme- 

are as follows : 

A. Variable - that which is to be changed or altered by 

a program for the target population, whether students, 
teachers, parents, etc. 

B. Variable Indicator - how. to measure or describe the 

variable. 

C. Goal - a statement of the desired change in a variable 
which a program is to affect. 

D. - Program T-echniques - the activities to^e carried out 
. j.n order to accomplish a" specific goal. 

(3) 
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E. Teciinique Indicator - indication or evidence that the 
technique has been carried out but not an indicator 
that the goal has been accomplished. 
An illustration o£ the use o£ these five evaluation elements 
is displayed in_ Chart I. 

Evaluation XiW/dsZ concentrate on the results o£ educational 
activities rather than on' the activities themselves. The final 
success of any educational activity must be measured against the 
re'sults it was designed- to produce. 
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SYSTEMS A??;<OAC;i TO EVALUATION 
Euucdt*ona* Sorviccs, luc 
WiiCO, Toxas 



Selection of variables to 
be changed, c.^. student 
bohjavior or knowledge 



Specification of a &oal 
for each variable 



Dcviclopmcal of indicators 
(observations or measures) 
for each goal 



Concurrent plan of 
educational prograin(&) 
designed to reach 
each goal 



Put program in 
operation 




/ 
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EVALUATION SCHEME ILLUSTRATION 



Variable 

Level of 

reading 

ability 



Var i able 
Indicator 



Scores on a 
standar i zed 
reading test 



Increase the 
reading 
abili ti^lf'^Tor 
200 fourth ^ 
grade students , 
who are 
ident i f ied as 
"slow readers" 
aS indicated by 
improvements in 
reading scores 



Program 
Technique(s) 



(5) _ 



carry out 
a special 
in-service 
training 
program tor 
10 selected 
fourth grai3e 
-teachers in 
assisting \ 
"slow reade^Vs" 

carry out a \ 
id-month 
program oC 
individual iz^d 
reading / | . 
instruction , 
for 200'^^fourth 
grade students 



JTechnique 
indicator 



Number of 
teachers 
complet in 
the progr 



Number o 

students 

involved 

in an 

individu 

reading 

program 
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EVALUATION SCHEME ILLUSTR-MION 



•iable % 
i cator 



)res on a - 
andarized ^. 
iding test 



Goal 



^Program 
Technique ($) 



Increase the 
read-ing 
abi lit ies for 
200 fourth 
grade students, 
who are 
identified as' 
•'slow readers'* 
as indicated "by 
improvements in 
reading scores 



carry out 
a special . 
in-service 
trainings 
prograisi. for 
10 selected - 
fourth grade 
teachers in 
assisting 
"slow readers" 

-carry out a • 
lO-mbnth 
program' of ^ 
individualize^ 
reading * 
instruction 
for 200 fourth 
grade, students 



Technique 
indicator 



Number of 
teachers 
completing 
the program 



Number of 
students 
invol v^a 
in an 

individual i zed 
reading 
prrgram. - 



(5) 
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Chapter il * 

Histopy and BackgrouDd ^ ^ 

\; ; ' . . ■ 

The Guided Occupational Orientation Project (GOOP) established 
by the Syracuse' City School Dis\:^ict (SCSD) of Syracuse, New Tork, 
is funded under Part D of PL-576 fbr Exemplary Projects 'in Voca- 
tional Education. • * 

At the time of this report, the projjsct has completed its 
second year under federal funding arrangjnents . Due to the fact 
that the federal funding cycle for this project i% February 1 to 
January 31, two compJ.ete school years are not represented in this 
report. / " 

o 

O 

In June, 1971; SCSD and Educational Services, Incorporated- 

♦ 

(ESI) of Waco, Texas, entered into a memorandum of arrang^ement 
v/hereby ESI agreed to provide an independent, third-party^ evaluation 
of GOOP as required by federal funding regulations- 

Since that time, ESI has conducted interim, evaluation s^tudies 
^i'n late June of 1971 *and 1972 to- study data representative of com- 
plete school years- Results of^ the^e interim evaluation? have been 
shared with GOOP project administrators and have served as the 
basis for some redirection of various project activities in sue- 

o 

ceeding school years. Through these interim reports^ GOOP project 
adminits trators have been afforded the opportunity to maximize 
utilization of evaluation in decision-making procedures. 
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GOQP £ua4ed and operated, involves -^11 three grade -group 
populations elementary, Junior high school, and senior high 
school. The complej^ities o.£ this 4)r0 j ect^ involving large numbers 
o£ students and faculty' members and afff?ctin^ diverse areas erf the 
total educational • progi-am-of, SCSD, l^came unwieldy and confusing. 

■ f • ' ' ' ' . 

'*' . ' * , 

both to GOOP administrators and administrators of oth6r activities 
0/ SCSD. This resulted in the development of, an overall Master 
Plan, an exercise in planning which involved ESI personnel working, 
closely with various administrators and GOOP staff members. 

The Master Plan is displayed in Chart'II.. It is thelThder- 
standing of the evaiuators that maximum uti/lization of this 
planning documen^ lias improved the understanding of the overall 
project design and flow for staff members] faculty members, and 
administrjators . . 

In January of 1972 and 1973, ESI completed formal: .eiJ-aluati on 
reports concerning assessment of all projelct activities throi^gh ^ 
those d.^tes.''^ Some early data, amissed prior to the association^ 

of ESI with the project, resulted from inslrruinents and procedures 

V • 

developed -by the piroject' staff. Instruments and procedures uti- 
lized for°,data gatfiering during the 1971-72 and 19T2- 73 , school • 
years reflect cooperative planning by GOOP staff members and TBSI , 

personnel. j . ^ 

The task of p-iiepar-ing a formal evaluation report at mid-point 
of the school year has pi-esented problems for the evaiuators. This 

See .Holder and Belcher ,. "Evaluation Report of the Guided Occupational 
"Orientation Project", February 1, 1972.- 
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report reflects cpmplete^^^'e and posy 4ata analysis fo:^ the 1971- 
' 12 school yea,r"Birrcomplete data fori the/l972-73 school year 

/ // 

will not be available until June, ^1/973/ 

The Master Plan disp,_lay.s ^ the /overall , long-range mission\o£ 
GOOP : ^'Increase the o.ccupationaly^areness and facilitate enti^aiice 
'into- the world- of worl^ for stud^?s rn the Syracuse Cit^ Schopl 
District Grades 5-12". This mis.Sion is extended into th^e three 
. .major' gradfe groupings With-^TOe following goals: 

*A, Elementary: Awareness - Increase the awareness of 
students of the /world 'b£ work an^d its composition; 
^ B. Junior High S^ool: Orientation and Explorati^on - 
Increase the orientation of Junior "High School 
students- an4 begin exploration of the work opportunities 
^ and necessary prepara-tion. , ■ ' 

C. Senior- High School: Job Training and Placement - ' 
Enable individualized career preparation and 



implementation of career plan. ■ - ^ 

•Each group goal is then supported ^ separate grade level . 
go^s wipr^'appropriate variables to support each. These grajie 
level goals are presented and discussed in Chapters, III, IV and 
V o£ the report. • ' , ' , - 
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I CHART n 

MASTER PLAN 

/^Guided Occupational Orientation Project 

Mission : Increase the occupational awareness and 
facilitate entrance into world of works 
for students iiv i^yracuse City School 
District Crade 5-12 



Elementary 



Jtinior High School 



Awareness 

Increase the awareness of 
students of the world of 
work and its composition. * 



Sth Grade 



'•What is the 
World of 
Work" 



Varjljtble 

Awarenesss 
of World 
of Work 



6th Grade 



•*What makes 
up the World 
of W^rk" 



;5 



Variables 

(a) Awareness of^ 
make up of 
World of Work 

(b) Attitude about 
; \yorld of Work 



(c) Awareness of 
manipulative 
l^lls.used 
occupations 



Orientation and Exploration 

Increase the orieatation of junior high 
school students and begin exploration of 
work opportunities and necessary preparations. 



7th Grad6 



Sth Grade 



9th Grade 



"Whp^am I 
andjiow do 
4/fit in the 
Wdrldof Work" 



•'What are my 
opportunities 

in, the world 
of work which 

relate to my 
interests" 



**What is my career 

preference and 
how do I iJrepare" 



Variables 

(a) Exploration.. 
' of interests, 

abilities, 
values, and 
needs. 

(b) Exploration 
of economic 
system' . 

(c) Exploration 

of opportunities , 
in Working World 



(a) 



Variables. 

Amount of 
exploration 
of career 
clusters 



Variables - 

Completion of 
a career plan 



(b) Exploration 
' of components ' 
of career 
' planning ' 
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(a) Initiay 
implemen 

(b) Mid-yetr 
assessmen 

(c) End-of-ye 
direction 



Senior High School 



Job Training and Placement 

Enable individualized career 
preparation and implementation 
of career plan. 



10th G^ade 



••What are-Tirst 
steps in 
training program?" 
, and "Should I 

alter my plan?" 



11th & 12th Grades 



"How do I 
complete my 
training 
program" 



Final Goal 



Post -graduation 
plan for career 
development with 
alternatives 



Variables 

(a) Initial 
implementation 

(b) Mid -year 
assessment 



(c) End-of-year 
direction 



Variables 



(a) 



Development of 
flnal decisions 
about post- 
graduation 
action 



(b) 
(c) 



Placement 

Options for 
career as 
related to 
interest and 
skills 
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Chapter III 
' Evaluation of the Elementary. Program 



All 
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The elementary segment of the Guided Occupatiaf.al Orientation 
Project is operating- under the general goal of awareness - "increas 
ing the awareness of students of the world of vork ^nd its compo- 
sition." 

The fifth grade program phase was Initiated in September, 1! 
and deals with the specific goal of ".What is the World of Work?'/ 
A special social studies unit, "The World of Work," was compili 
from existing sixth grade material. This unit was introduced /t( 
all fifth grade teachers, during special ^meetings and will be uti-' 
lized in ^4^classrooms in 34 elementary schools (31 public anji 3 
non-public) and involves 2,444 students during the 1972-73 school 
year. ' / 

The 'sixth grade program phase operates undfer the specific 
goal of "Wh^t Makes Up the World of Work?" andihas been functional 
since November, 1970 . Early stage;5 of these sixth grade activities 
were not supported through federal funds. This support w^^ initiated 
in February, 1971. Thus, the sixth grade program reflects' the longest 
time span of experience in GOOP. 

During the period from November ^ 1970 to June, 197l|| 1,260 
sixth grade s-tudents representing 17 schools, participateii in the 
program. Erom September, 1971 through June, 1972, l,75q sixth 
grade students representing 23 schools (18 public and 5 /hon-public) 



participated. Jt is projected that 3f, 794 students representing 
29 schoolfe (24 public and 5 non-public) will participate in the 
program during; the 1972-73 school year. 

The^program basically includes the following components: 
Two weeks orientation o£ the world o£ work (review o£ 
fifth grade materials in 1972-73 school year). 
Two field trips to -the Career Center located in the 
George Washington School to, visit and use display 
booths explaining a variety o'f vocational and 
industrial areas. - " - 

Three weeks of individualized study of selected • 

careers coupled with class activities. 

Two days of a' skill program to enable students to 

experience occupational tasks* and role playing. 
Goals for Fifth Grade Students 

1. Increase the level of awareness of broad families of 
jobs, employment requirements, personal abilities and 
interests in relation to various occupations by higher 
mean scores on post-test for ''An Introduction to the 
World of -Work" as compared to the pre-test. 
^. a. Variable - level of awareness of broad f9.milies 

of occupations, requirements for employment, per- 
sonal abilities and interests ^in relation to various 
occupations . 

. (12) 



■17 



Indicator - pre and ^)ost test for program, ''An ' ^ 
Introduction to th-e World of Work/*' 
^This goal and the educational activities .for f if thr grade . are 
the first steps into GOOP for elementary students. 

Fifth grade ^students completed an instrument designed to- 
measure general awareness of the world of work,* before participat' 
*ing in two weeks of classroom orientation and- after cTompletion of 
the orientation. - ' - 

At the time of this report 282 fifth grade students have com- 
pleted this orientation in the 1972-73' school year. A twenty 
percent systematic sample of these students was drawn for project 
evaluation/ Pre and post test mean scores are shown in Table 1» 

Table 1. ' ' . * ° 



• 


1972- 


•73 








Pre 




Post 


Difference 




Test 




Test ■ 


(Gain) 


Means • ' - 


15.98 


'19.37 


3.39 


Sample Size 


53 


53 





Pre and post test means for the 53 sampled students were 

15.98 and 19.37, a positive gain of 3.39 positive points on the 

* The instrument was designed and pilot tested during the 1972- 
73 school year. The maximum score possible is 27. 

' ^ /' (13) - - 
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averag/per- stud/nt. Student awareness of the "world^ o£ work' 



is >ii^reasing/in the desired direction according to thia test.- 

B. Goals fot Teachers . 

1. Participate^ teachers will return class lists indicat- 
ing involvementNof 100 percent of their students and a 
minimum of ten clas^ hours spent on the GOOP program. 

a. Variable - utilization of program and materials 

r 

made available by the^ CaT.ee r Center. ' ' i 

b. Indicator - number of class lists indicating 
grades returned to Career Center and hours 
involved. • < . 

Fourteen fifth grade teachers had .completed the program at the 
ti^ne of this report.. All oCl.OOl) of the teachers returned class 
reports indicating that all students had participated. The average 
number o~f hours per classroom was 16.99. 

C. Goals for, Parents 

1. Parents or guardians are to act as a resource person to 
• be interviewed, 
^a; Variable - .level of interest' and participatig^l in 

classroom program. - 
b. Indicator - completion of interview of '^parent or 
guardian, included in the world 6£ worl<f stu^Jy. 
, All stude^'ta^in the program to date have . completed an inter- 

view with parents or guardian. ^ 

Business and industry- are not involved: in fi%th gt^de program. 

\ (14) \ / 

.19 
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Sixth Gradq Students * " . 

Duri-ng the 1971-72 school year, sixth gr^de students represented 
the primary elementary level target population for career education. 
For ;^he ' 1972-73 school year, the same program with only minor mod- 
ification has been continued. 

T-lie 1971-72 goal of i-ncrease^d 'awareness of th6 world of work 
has been expanded to two more specific goals in 1972-73. At the 
time o£ this report, 300 sixth grade students have completed a 
careo^r education unit in .the 1972-73 school yea-r. 'Followirtg are • 
evaluation goals for 1971-72 only, and both school >years . ^ 

1. Increase the awareness of sixth grade students concerning 

the world Of work. ' (1971-72 only) . 

"a. Variable - level of vork awareness 

b. I'ndicator - scores on "Awareness of Worl^.^ Work" 
/ ■ 
^(Orientation) test. 

This goal and variable reflect the first step i^ito the complete 

GOOP for elementary students. For 1971-72 the only grade involved 

was sixth' grade, the same grade involved inll970-71. Students 

complete two weeks of classroom orientation in this phase. The 

test is designed to measure the student's general awareness of the ^ 

world of work. 

Pre And post test scores for an approximate ten percent random 

> 

sample of students shown in Table 2 are 15.59 and 18.6. Student 
awareness appears to be increasing (as indicated by this test) as 
a result of the two week orientation. No statistical test of i 
significance was calculated. Use of number of career areas iden- 
tified by students was developed- as an additional indicator because 

^0 



of questioas^ about'* Validity o£ , measurement even though this indicator 
was i-ncreased following the orientation phase. 



Table 2 ■ 
Career Ori?entati0n Test 




Means 


l'5.59 


18.6 


Sample Size 


\ 171 


1 

146 . ■ 



As shown below, over ninety percent of sixth grade teachers 
rated this introductory phase as 'Valuable" or above. 
Absolutely Esjsential 
Very Valuable 



Valuable 

Not Very Valuable 
Waste o£ Time 



14.8% 
44«r7%. 
3.5% 
OS 



mc 



2. Increase the level of awareness of the fundamental concepts 
and procedures related to the world of worl^;a5 indi'cated 
by higher mean scores on Career Educa-^^^on jihit Pj^^-test 
as compared to srcores on pre-test. -(197^-73 oaiyO 

a. Variable- level of awareness of the fundamental 
concepts and procedures related to the world of 
work. 

b. Indicator - pre-post tes.t scores on unit, Career ' 

♦ » 
Education Grade Six. 

C16> 



c 

\ 



twenty percent systematic sample o£ sixth grade students 
—were tested as part o£ their Career . Education Unit as to their 
career -^'concept understanding. Pre and post test mean scores as 
^shown in Table 3 are 6.88 and . 10. 84. The g.ain (5.95)' is in the 
desired' direction and .supports intended- cbnc'ept gain. 



Table 3 



Sixth Grade 


Scores on 


Career 


'Concept Test ' . . 




1972 


-73 








> 

Pre 
Test 


Post 
Test 




Difference 
(Gain) 


Mean- Scores 


6.88 


IX). 84 




Total Students 


70 


70 


5.95 j 
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3. Increase the student's knowledge o£ the career ladder 
concept in local business and industry as indicated by 
higher mean scores on a true-false post-test as comp^a'red 
to pre-test scores. (1972-73 only) 

a. Variable - knowledge o£ innumerable career opportu- 
nities and respective occupational ladder concepts^ in 
^ * local business and industry. 

"b. Indicator - tru^-£alse scores before and after. visit 
to career center. 

•4 

One question raised in ESI's previous evaluation was the 
inability to assess the specific impact of student use of career 
booths describing job families available to the Syracuse area. A 
true-false test with 10 questions was conducted by the Career Center 
and administered to, all students participating in the two hour tour ^ 
of th? center including use of the booths. Pre and post test mean 



(17) 



scores, shown in Table 4 weife 6.13 4nd 6.89, a gain o£ only .76 



points . ' ' \\ 

'^-'^^ The school year is only half oveV anS all target sixth graders 
have'' hot completed the program consequently no conclusions can be 
drawn from this small difference. The ) difference may be, resulting 
from inadequate measurement of .dvarig^ ; non- representation of class- 
rooms participating to date; and/<Jr failure<of career booths to 
producd tha desired change in- inf^ormatiOTj'. End of year data on 

^This test will enable a more definite determination. if there is" ^ 
a problem, • * . ^ ' ' 



table 4 

Sixth firadg Career Ladder Concept Scores 



1972-73 



Pre 
Test 



Post Difference 
Test - (Gain) 



Mean Scores 


6.13 


._6.89 \ 


■ .76 


Total' ^Students 


■ 300 


' 300' 


r 



iTVGxease positive attitudes toward wo'rld of woi:k (occu- 
pations and careers) as indicated by statistically 
significant mean scores on an attitude ques tionnaire% 
(1971-72 only) 

a. Variable attitudes concerning world of work. 

b. Indicator - a'^composi^te attitud-e score based on five 
, selected attitudes statements. 

' (18)/ 
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One assumption of the elementary program has been that students'^ 
Lack a positive attitude toward careers and occupations. An attitude 
composite score made up of responses to five statements was, used 
with "0" as the most negative and ''5V as most positive indicato^rs 
of attitudes. Pre and post tes't scores are shown in Table 5 for 
an approximate ten percent sample. While scores do increase, the 
gain IS only .33 overall. 

Since mid-year, evaluations also indicate only slight gain in 
already positive attitudes. Two sixth grade classrooms completed 
seventh grade level attitude questionnaires to determine if less 
potential attitudes may be present on higher level career concepts. 
Composite scores (based on five selected statements) as ^shown in 
Table 6 are similar to previous findings even though the gain is 
less. * . ^ ^ fl 



Table 5 

Sixth Grade Composite ' Attitude Scores 
• ' 1971^72 
Pre Test 



Post Test 



Sample Group 


Sample 
Size 


Mean 


Sample 
Size ^ 


Mean 


Program participants 








1 


using 6th grade 






146 


4.08 


questionnaire 


171 


; 3.75 
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. Table 6 \ 

« 

Sixth Grade Attitude Sample on Seventh Grade Qges tionnaire 



1971-72 . 



Pre Test 



Post Test 



Sc'mple Group 


S^l^lple 
Size 


Mean 


Sample 
Size 


Mean 




Participants usirfg 
7th grade question- 
naire 


13 


3.77 


15 


3.87 





In general it would, appear that students do have a positive 
orientation toward the world o£ work at least as indicated by the 
Statements used. The findings -suggest that an assumption o£ lack 
of positive attitude is involved and should be reyise'd. If this 
variable is important to the elementary program, then mare specific 
Student orientations should be flfeasured or the variable dropped from 
consideration. We believe that the general positive orientation of 
American society to work (ilo specific careers or' occupations) will 
be held by the young (certainly at this age) even though uninformed 
about the makeup or elements in the world of. work. Attitude change 
at elementary level may be less Important than 'more» specif ic atten- 
tion to increased awareness and in-depth exploration.''^ 

5. Increase awareness of make-up of career areas foT at iea'^t 
three occupations. (1971-72 and 1972-73) ^\ 
a. Variable - awareness of occupational areas. ^ 

* As a result o£ end o£ school yeaf evaluation by EST, this .variable 
was dropped in 1972-73. 
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• b. .Indicator - number' of correct identifications of 

characteristics of caree^r. areas on a standard test. 
Th"*S pre aiid post test results for a sample of. sixth grade 
students for igyi-Ji , and 1972-73 ,Cto date) are shown in Table 7. 
Student awareness did increase for 'students' in both school yea_rs 
with a gain of 2.28 and 1.93 points respectively. The program does 
affect the irtformatibn level and understanding of the world- of work 
Wld by siifcth g.rade students as indicated by this teSt. These ^ 
Results are" similar to those for tha. 1970-71 school y©ar where 800 
-sampled sixth grad'e students had improved scores for- the following: 

* ^ ■ . 

'--^.(a) A career concept matching test (a reduction in near 
number ,d£ errors from 7,75 to 4 4:^67) to determine 
ah^ility to match concepts related to wdtking . with 
possi,ble defini tion ; 
' -I (b) A career ladder test (a reduction in mean number o£ 
errors from 2.83 to 1.58) to deteinnine ability to ^ 
'match job's with the level of education most qften 
required. Career Study "i/^hievejiient Test had an 
increase 'in mean scores from 7.35 to 8.*67 for a 
random sample of^^ 55 students who had complete^ the 
programs during^ the first half o£ -the 1971-72 school 
\ year. ^ • 

(21) 
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Table 7 



Sixth Grade 


Student 


Awajreness o£ 


the Make^ip of 


the World 


of- 


Work 






\ 1971-72 


1972-73 


(To Date) . • 






Pre Test 


Post Test 


Pre Test 




P'ost .Test 


Mean 




6.4 


8.68 


6 . 81 


8.75 1 


Sample Size 


171 ; 


• 146 


138 


138 1 



. Gonseq^uently we have found agreemeTit ifi all measures o£^ s W^J/ifent 
gains in occupatipn^i career awareness. The reiiability of instru 
ments is demonstrated by consistent production of similar res^ulits. 
No. tests of statistical significant. differences were dondjucted as 
resou^ices w^Q^not . available to carry out the data analysis but all 
test changes were in ,the desirable directions as expressed by goals 
6, Increase st;udent awareness of the skills required for- 
/ ' . 'related occupations as indicated by , statistically "sig- 
nificant differences i-n pre and post tests of skill 
awareness as a result of working in the skill trainer 
> ■ van. (1.971-72 and 1972- 73)^ ^ . 

a. Variable - awareness of occupational' skills. 

b. . Indicator - scores on an occupational skill awareness 

. * • • • 

. 'test. ' . " ; 

One phase of the elementary progjam has been two days of work 
iji the skill trainer (a van equipped , with tools, projects and equip- 
ment from a variety of .occupational : aJi!^eas) . Evaluation* of this phase 
is based on increase in student dw^reness .of skill requirements as 

. . ^ ^ / ' ■ ' 4 

well as teacher assessment. * . 

; ■ • • (22) ; 
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Pre and post test mean scores for skill awareness shown in 
Table 8 for sixth grade students, in 1971-72 were S.36 and 11.-3 
and fpr' the 197'2-73 (to date) i^ere 3.74 and 8 . 1^8 Student aware- 
ness o£ skills required for occupations has increased. Teacher 
evaluations for the skill trainer for" 1971-72 are shown in Table 
9.. All teachers rated the experience as at least ''valuable*' or 
higher. Close to fifty percent rated the experience /'absolutely 
essential''. . The van appears to provide important tactile and 
life . simulating experiences for students where tools and equip- 
ment can bre used to carry out a project. Students are able tC 
touch, feel, and maiiip^ulate objects related to occupations,* an 
important compl ement tq_ verbal descriptious as written material. 



Table 8 



Skill Area Tests 
1971-72 



1972-73 





Pre Test 


Post Test 


Pre Test . 


Post Test 


Me an^ 


5.36 k 


11.3 


3 . 74 


8.18 


Sample Size 


125 


104 . - 


' 243 


149 ' 



Table 9 

Teacher Evaluation of Skill Tyainer 



Teacher Evaluations 
«> 


• 1971-72 






Absolutely Essential 


42'.4% • 


Very Valuable * V^"*^ 


34.6% , 


Val^uable^ » > 


23.0%'' ' 


Not Very Valuable 


0.0% 






Waste of Time': — ^ 'r- . ^ - - 


■0.0% 
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Six-th grade teachers were asked to as^sess changes in a random 
sample of their class during the program. In 1971-72 results of 
their ass^esXment are shown in Table 10. The program is viewed as 
improving c the motivation for SA-s o£ the students and improving the 
workd-^abits of 48.5V of the students. 



Table 10 

Teacher Assessment of Sixth Grade Student Changes Durj!1fig Program 



1971-72 



Student Behavior 



Improved 



Stayed 
the s ame 



Deteriorated 



Moti'vation 


541 


36%" 


10% 


'Work .Habits 


48.5% ' , 


4'4%' 


'7.5% 



"Students Evaluated 



200 



B. Goals for Teachers • ' " 

\ J. Teachers will permit and encourage individualized career 
study in classrooms. ^Specifically the mean number of 
individual career S/tudy projects will be at least 2.0 

per child.^. ^ / 

/ - 

a. Variable - levfel of implementa'ti-on' of individualized 

- ' ■ ■ 

' instruction in "classroom. 
5 b. Indicator - average number of individual career study 

\ • ^ packets per student completed. 

St^ddnts are. to study separate career areas on their own follow- 
ing their visit to the Career Center. According to program philosophy, 
this study both enables a student to pursue in-depth car.eers of ^ 
- • C24) , • - 
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i-nterest and to study on their own ,at their own rate, an important 
- concern 6£* the ov^erall program. For the two previous com^plete 
school ye'ars and during 1972-73. to date, actual averages have 
exceeded the minimum a$ shown in Table 11 while likely not statis-^ 
t'ically significant differences are reflected. However, the.jnean 
' number each-year lias fallen. " Independent self - ins truction may be 
decreasing. ^ - ' 



Table '11 

Individual Career Study Packi^ts Completed 



1970-71 



1971-72 



\ 



1972-73 



Goal 



Me an numb e r ■ 
per student 


2 . 4'3 


2.17 ■ 


\ 

2.01 


Minimum 


Sample Si-ze 


871 
Students 


- • ^171 
Students. 


\, V 12 - 
\ Classrooms 


1 \ 



This d^cre^se'may ^so correlate with tea^^hers' preferences. 

Teachers, while rating the^^ individual career studies as valuable, 

.* * » 

\' 

are not as enthusiastic about this part of the sixth grade^^program 
as other parts . 

For the school year' 1971- 72 17 . 3?;' consider this **absolxitely 
essential'* compared with 37% and 42.4% for the same rating for 
the orientation study and skill trainer.' See Table 12. 
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' * Table 12 . . 

. Teacher Ratings on^ Individual Career Studies 

Teachers' Evaluations Percentage . 



Absolutely Essential 


17.3% 


Very Valuable 


34.5% 


Valuable 


44.7% 


Not Very Valuable 


3.5% 


Waste o£ Time 


.0.0% • ^ ' 


C. Goals ^£or Business' and Industry • . ' 



i. Local business and industry , will be involved in career, 
orientation 'for elementary students . Specifically, those 
' . contacted will maintain at least 17 career exhibit booths 

or at least four career' penter representatives will' visit 
each classroom or allow field visits for at least eighty 
percent of those capable of admitting young children to 
plants and offices. 

s.'. Variable - level of involvement . of business and 

^ industry. * 
b. Indi'cator - percentage of those contacted who had 

a career^ booth made classroom units ; and/or allowed * 
students to m^ake field trips. 
The seventeen career, bootlis were maintained throughout the 
1971-72 school year and continued in the . 1972-73 year . On thej. 
average, the number of visits by career representatives for 
, ® (26) * - . 
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' 1971-72 and 1.972-73 (to date) was 3.9 and 4.46 per classroom. 
Elementary staff members obseAed that such visits are affected 
by limited time to initiate and make arrangements for such visits. 
Consequently, responsiblity for making arrangments was left to ^ 
teachers who often did not follow through. Over eighty percent 
of the business and industries able to do so allowed field trips. 
The only area where student field trips were not encouraged was 
at local hospitals. The average number of 'fie;Ld ' trips per class- 
room was 4.76 for the 13 sixth ^grade classrooms completing the- 
program'in' 1972-73 at the time of this report. 
D. Goals for Parents . • * 

1. At least ten percent of parents per classroom will 
participate in the program in some way. 
.a. Variable - level of interest and amount of partic- 
ipation by parents.^ \ ^ / ' , " ' 
b. IndicatQ'r - number of classroom visits per parents, 
"listings of parental contributions and percent of 
parents visiting" clas.sroomS . ' ' , 
Evidence obtained by elementary Career program staff from 
teacher anecdotal reports in 1971-72 indicates that where parents 
visited the classroom, usually tl^ey' visited only once. However,, 
for 1972-73 over fifty percent of the parents visited classrooms. 
As a par.t of career orientation all parents were interviewed by 
students concerning their occupation. Parents served as resource 
♦personnel to describe' their jobs to classes, assisted with super-vis.ion 
of fi^ld trips, and visited the. career center. About se^en percent 
visited the skill trainer, according to elementary staff members. 

.(27) 
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E. Elementary Summary • '^f 

Chart III summarizes the variables for each j^arge|^, group 
students, teachers, parents and industry the corresponding 
desired effect for GOOP, and. the agtual effect td 4afe, based 
on most current data. * ^ - 

The fifth grade program, td date, is reali^zing its goals and 
appears , to be operating smoothly. This program, in ESI 's judge- 
ment has benefited greatly by the experiences with sixth grade 
students in the prior two-school years. 

For the sixth grade, knowledge and awareness variables for 
students ^have increased as desired, with the exception of attitudes 
about world of work (which has subsequently been dropped) and 
knowledge of career oppo3>tunities based on a visit to the career 
booths. Individualized instruction is close to the minimum- 
indicator and there is evidence of a decrease. Attention should 
be paid' to increasing extent of individualized instruction if this 
variable is important to the program. Results for business and 
industry and parents are meeting expectations. 
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Chart III 
Elementary Evaluation Summary 



Variable 


Desired Effect 
By GOOP 


Actual Effect 
(Based on most 
current data) 


1. 


Pifth grade Students 
Awareness o£ world 
of work 


Increase 




Increased 




Fifth Grade Teachers 
Utilization of maXe- . 

/iais lOO'o of teachers 


100% 




Fifth Grade Parents 

Parental' interviews All students 


complete ^^^^ f* 




.qiYth Hrade Students 






• 


a * 


Awareness of world 

of work - Increase 




Substantial 
. ^ Increase 






Awareness of fun- J 
damental concepts Increase 




i 

Increase 




c . 


Knowledge of career 
opportunities Increase 




^ Very Small (To 
Date) Increase 




d. 


Pqsitive^ attitudes . 

toward world of ^ 

work ' Increase 




Shift Increase • 
. Variable Dropped 
1972-73 




e . 


Awareness of make- 
up of these career 
areas Increase 




Marked 
Increase 




f . 


Awareness of career 

skills Increase 




Substantial 
Increase 


III. 


Sixth Grade Teachers 
Level of individualized 
instruction permitted Minimum of 2 

projects for 


individual At least minimum 
students for three years 



0" 
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Variable 


Desired Effect 


• Actual Effect • 


IV. Sixth Grade Business ^ • 
and Industry , ^ ' 




Level of 
involvement " 


• Maintain 'career booths , 
visit classrooms , - and 
permit field trips 


Good dnvolvement • 
and support 


V. Sixth Grade Parents • 




^ Level o£ interest 
and participation 


At" least 10% . 
participation 


Moore than minimum 
participati<on 



4 



(303 
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■ ' • I diopter IV 

.■I Evaluation of Junior High Program 

. The junior high scheol segment of the Guided Occupational . 
Orientation Project is directed toward the achievement of the 
goal - "Increase the orien'tation of^unior high school students 
and begin exploration of wxjrk opportunities and necess'ary pre- 

♦ 

paration." 

This general goal is individualized to seventh, eighth, and, 
ninth grade .activities , ^ ^ - ^ 

The junior high school program segment was formally imple-. . 
''m'ented in all junior high schools in the dist/ict iTi" September , 
1971. The program segment was dependent on curriculum units . 
developed by writing teams the previous summer. 

No curriculum materials were pilot-tested in the 1971-72 
school year. Also, composition of first-year writing ' teams did 
not provide adequate checks ai^id balances. Curriculum units were 
introduced on a mas'S basis into all junior high schools. Some 
material was not appropriate to grade levels, some suggested 
activities were not realistic, and some lis,ted resources were 

D 

not readily available. This served to intensify teachers' re- 
sistance to a new program. ^ . 

However, these problems were flagged 'i-rt January, 1972 ;^fduring 
• (31) 
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evaluation procedures • New 'curriculum units ^were designed by 
exp^iided and balance)! writing teams during the summer of 1972., 
All units w^ere pilot-tested and are l?eing introduced into other 
Junior l^igh 'schools one at a. time. . 

The, program has benefited from one false s tarlf^ty -the sufe-^ 
seq^ueat - revisipn of techtiiques. ' . ' ^ ' ' 

• The junior high school' program is directed by three Occupa- 
tianal Resource Specialists (ORS) whose tasks include introducing 
the program to administrators and faculty members of each school. : 
They also assist individual teachers and serve a,s resource co-' 
ordinators. The enthusiasm for ^he program evidenced by these* 
three young educators has been very "^significant inoovercoming 
early ^problems . ^ ^ 

For the 1972-73 school year, 6,048 junior high school students 
distributed among nine schools are involved iA the program. 

Followii^g are goals covering s;fe^dents , teacherS, parents and' 
business and induS try^^spanning^eventh , eighthyvand ninth grades. 
For the 1971-72 school year, goals onlyTxela^^ to stadents. All 
grade levels participated in the same ca/eer program, thus all 
had the same^,,g^ls. . . 

Firsi^ goals and results £^m 197?.-72 are discussed, followed 
by 1972-73. - - . 

A. Goals for Students ,(1971-72 only) 

1. Increase career choice awareness for seventh, eighth, and 
ninth grade students. 

a. Variable - awareness of career ^oices% 

(32) * 
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'b. Indicator - listing o:^ alternative career choices. 
One goal o£ the program is to increase the career choices of 
which students 'are aware. Students have impressions about possible 
careers or occupations, many o£ Vhich are valid, while others (as 
indicated by pre program listings} do not exist or m^y describe 
nonexistent occupations. The mean. number of career choices by 
grades are shown in Table 13. Only ninth grade students had a 
significant increase i% alternatives identified, but a review of 
student questionnaires suggest that student choices become more 
serious and more informed while the number of choices may have . 
stayed the same or slightly increased. 



Table 13 

Career Alternatives Identified by Junior High School Students 

1971-72 

7th Grade 8th Grade 9th Grade 



Fall Means 


' . 8.04 


9.67 


^8.95 


Spring Means .> 


8.62, 


■ 9 ".17 


^6.74 
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2. Increase resource awareness as to the typ^s of resources' 
with'which students are familiar and hgive utilized, 
a. Variable - awst-reness 6,f resources for vocational 
guidance and career planning. 
" b. Indicator *- listing of resources. 

Career r inforraation.rSources * used by ' a random sample of all \ 
junior high school students are shown in Chart 4'. Generally, the 

■ * (33).^ 

• 38 • 



information sources which 'most co'ns^s'tently gaih in cited, use by 
students for career information are first, guida'nce counselors and 
second/ interviews withr employers. 

Teachers relatives , friencjs, libraries, l^ds and want ads , and 
field trips gain only in two of the three grade levels. The formal 
organizations representing employment or busmes^ svuaa as labor 
unions, chamber of commerce, ox civil s-ervice generally drop in 
cited use. Apparently, a result o/ the program was reductit)n in 
' ' the total number of career information sources for students with 
increases in cited use of those sources students found mast use-<- ^ 
ful during the program. Guidance counselors were primary ihfor-. \. 
/ mation contact and gains in cited usage reflect this. 

3. Students will increase career awareness as a result of ^ 
the program. 

a.* Variable - level of career awareness 
*b. Indicator - pre and post scores orx a 12 item test ^ 
' of informatit)n about careers and career development*. * 
■ . Frequency of correct answers fo.r the three grades are shown 
in Ch^rt 5. Spring mean scores are substantially" higher than fall 
scores for all grades. Career awareness, as indicated by Ithi^ .test 

* r 

markedly increased during the 1971-72 school Year. 
^ ^ 4. Students will increase- positive attitudes towardxareer 

planning. . r * . 

V 

a. Variable - level of positive attitudes toward career 



planning . 
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Chart 4 

Career Information Sources Used. by Junior High School Students 



Percentage o&^ Sample Indicating Use 



Information Source 



Fall Spring Fall Spi/itig 



7 th Grade^ 



8th Grade 



Fall Spring 
9th Grade • 



1. Guidance Counselors 


' 18.8 . *. 37.5 


44.1' 66.1 


•57.1 6 2.7 


2. Teachers 


48.6 51.5 


41.2 40.5 


36.5 45.8 


3. Relatives • ^ 


k 55.6 '54.4 


52.2 59.5 • 


65.9 69.5 


4. Fjriends 


04.6 55.9 


'54.4 6 7. -8 


.73.0 ^78.8 


5. Field Trips 


•33f3 41.2' 


25.7 53.9 


30.2 , 28.8 


6 . Ads ano^ })Vant Ads 


^ U . 0 4 Z . 0 


4^.U 40.0 


"iO 0 50.0 


9 

7. Interviews with 
\ Employers 


21.5 ' 29.4 


19.9 27.3 

/■I ■ ' ' 


2 7. 8 ..30.5 

TT^ 


a/. NY Employment Ag^n^cy 


12.5 12.5. 


23.5 ^ • -17.4 


\2.2 16.9 


(h • 0 yi yr *3 i" £> A rrfST^^^i P c 


9 7 lli B 


'20.6 13.2 


14.3 . 11. 0 


-j ' ■ 

1/0. Chamber of Comme/ce 


9.7 . 8.8 


14. D 4_ 5.8 


111 9: 5.9 


T — — ~ " 

/LI. Oivil 'Service 


6 ..9 9.. 5 


12.5^ 5.8 


— 

11.1 . 3.4 


'12. Labor Unions 


9.0 10,3 • . 


12,5 .4.1 


13.^ 6.8 


-£! ti^ . 

13. Professional 
Societies 


liri 11,0 


■12.5 7.4, 


15.9 - 7.6 


14.^ Colleges^. an^'^Schools 


23.6 . 19.1 


2i'. 3 • - 20.7 


27.0 16.9 


;15. Government Agencies 


9.7 8.1 * 


.16.2 1^.4 


" 15. 1 11.0 


16 . Libraries Schools , 
Books ^ etc. 


55.6 ' 68.4 


43.4 • 6 2.0 .■ 


^1.1 • ,55.9 


* • 

17 i "Military. Recruiting 


13.2 ■ , 5.9 


14.7 .10.7 


19.0 ■ : -4.2 
— ' '\ 



Total Students 



144. 
C35)- 



136 



126 



Chart '5 

7 -f- "U rt _^ _ T * ' 5 



Total Correct 
' Items 



"7th -Grade 
JPercenta'ge 
Fall Spring 



8.th Grade 
Percentage 
Fall Spring 



9th Grade 
Percentage 
- Fall- Spring 
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b. Indicator - composite score on five representativa 
questions reflecting facets o£ career planning. / 
Mean scores by grade level are shpwn ijt Table 14. The differences 
between fall and spring mean scores are insignificant, suggesting 
that either the- questions composing the ^individual sx:ores are not 
valid measures of attitudes or attitudes^ were not affected by the 
program. Since a score of 5.0 is the middle of the range, group ; 
mean scores arpund this point reflect' neither strongly negative or 
strongly positive attitudes. 



" Table, 1% . « ' , . ' 

Attitude S gores for Junior High SchooJ. ' Students 

* 19 71-72 " / * ■ 

7th Grade- 8th Grade 



9th Grade 



Fall Me^n Scores 


-3.48' 


3.50 


3. '60 ' * , 


Spring Mean Scores. 


3.38 


3.S9 


3.69 , .' 



"""^"^acher assessment of the junior' high school program is shown 
in Chart 6. Teachers were evenly^ divided in t^ieir beliefs about 
GOOP's fulfillment of student interest in career studies., but defi- 
nately disapproved of the appropriateness of. booklets for seventh 
grade students. They cited the need for revision of seventh grade 
materials and belieye4 time allotted insufficient for the 
material to be covered. . 

/ A majority o£ teachers said they felt, career studies to Jdc . 
nebe^sary except science teachers' where only twenty '.eight percent 

' . (37) , : 
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definitely were positive about career studie3. However, only - 
sixteen^ percent o£ the teachers said ''most*' .of the students showed 
interest in career studies. 

ESI * s assessment is that teacher recommendations for changes 
In the seventh "^rade program have been carried out in materials for 
1972-73. . ^ ' . • ■ [ 

* * 0 > 

B. Goals for Studehts'^lS^Z-^S) 

''^Mid year data for junior high, school students was not available 

for student goals but will^ be reviewed in the end of school year 

■ * 

evaluation. Goals, for students are as follow?: v. 

1. ..All junior high students increase their positive 
-attitudes toward career planning by a statistically 
significant difference between pre and post tes^ts^ ' ^ 

2. Increase resource awareness as indicated by a statis- 
tically significant difference reflecting resources 

. <• ' 

with which students >re familiar and/or have utilized. 

3. Students complete seventh grade with awareness of' ^ 
and' familiarity' with career cluster concept as \ 
demonstrated by s.tatis tically significant di*fferences , - 
on pre-post 'tests results.- 

4. Students increase awareness of jot^ emphasis as reflected 
by scores on'a* twenty item data, people, and things 
concept . ' 

5. Eighth grade students | by end of year, select one 
career cluster, for in-depth study during ninth' grade. 

.6. At least e'^ighljy five percent of s'^tudents will complete 

• ' ■ '(38) . [ 
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a written future education plan by the end of- the s;chool 
year in ninth grade. • . - " 

G. Goals for Teachers (1972-73) 

1. Participating teachers, on the average, will make at l^^fst 
seven requests and/or library check-outs of resource 
materials. • ' " 

a* Variable - utilization of resource materials made 
• ■ available by GQO?. 
b. Indicator - compilation of requests and qheck-outs 
by ORS team members. 
Number of reports by type- by . teachers is shown in Table 15. for 
1972-73 at the time of this report. 

' ■ . ' Table 15 . 



T.eacher Requests for Help and Materials 
197.2-73 CtQ date) 
Teacher Requests Number 



^1 — 1 

Fi'eld Tripa-^ 


21 


Speakers 


10 - ' 


Written Materials - 




Meetings (Spec. Problems)* 


ij ' ■ 


Other 


2 • < ■ , . 


AV Material 


25 


/ * Total 


70 
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, ' Chart 6 • . 

Teacher &valuatioa of Jui)ior High School Program 
. . 1971-72 



Yes 



.Assessment of Materials 



No 



INo Answer 
"Not Sure 



Do you feel: 

A. GOOP aided in fulfilling . 
students interest in 
career studies? , ^ 


36 


39 


/ 

25 


B. The booklets, used this 
year were appropriate- 
for 7th grade? 


18 




9 


' C. The 7th grade program 
materials should be 
' revised to provide 
more structure? 


61 


9 


30 


D. The content too exten- 
sive Sot 120 class hours? 


45 ' 


25 


30 



\\. Assessment of 'Career Study* 



-6 

Yes 



% 
No 



%No Answer 
Noti Sure i 



1. All teachers 


■ 60 


17 


23 


2. English 


68 


1 ^ 


16 


3. Social Studies 


54 


8 


38 


4. Math 


'80. 


20 


0 


5. Science 


28 


44 


2'8 



A. Do you as a teacher, feel career * studies are. necessary? (Above) 

B. How many students showed interest in career studies? . 



Most 



%Some 



I Very Few 



%No 

Answer 



16 



56 



25 



Total' Teachers RespondiAg - 47 
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Other teacher goals for 1972-73 £cr which no^dat^ was available 
at'.the time of this report are as follows: • • 

Every junior high Giassro.om teacher of Math, Science, 
* So'cial Studies , ''and English will participate in the program. 

^ Integration ofGOOP materials into standard curriculum 

offerings in all classrooms of participating junior high 
' schools . ' . , ' , 

For this evaluation report, over eighty, teachers from nine' 
junior high schools were asked to assess the accessibility and 
utility of career education resource kits. Jhis information Wil! 
be valuable in altering matfe.rial distribution -and teacher moti- 
vation activities during the remainder of the school' year aTnd 
modifying materials for 1973-74. Results are shown in Chart 7. 

Materials are generally perceived as available by teachers, but 
utility and helpfulness in teaching careear educati-on tiave lawer 
ratings. Only twenty two percent of the teachers report that they 
use material very regularly Qr regularly. But almost severity per- 
cent of those not now using materials plan to do so in the future. 
D. Goals for Parents (1972-73) ' . ' . . 

1. Disseminate information on 'a continuing basis to parents. 

a. Variable " parent exposure to information about career 
education. 

b. Indicator - number of letters, pamphlets, PTA appear- 
ances, and media articles related to GOOP. 

(41) 

46 



Total information output in 1972-73 under this goal is.shownr 
below: ' • " 

' * • 11 PTA presentations 

6 PTA handouts ' ' / 

6 Television news segments 
10 Newspaper articles 
1 Newspaper picture story (4 pages) * 
El Goals for Business and . Indus try ,(1972- 73) 

1. At least seventy five percent of business and industry - 
contacted by students, classes, and ORS s*taff react 
favorably to inquiry. 

a. Variable - extent df community openness to student 
inquiries. 

♦ 

b. Indicator - number of acceptainces of student or class 
'inquiries by business and industry. 

Number df requests and contacts in 1972-73 at time of this ^ 
report are as follows: 

30 Requests were - favorable 

3 Requests were unfavorable 
26 New contacts made and added to resource file 
Two schools requested and were granted one-half day in-service 
workshops funded by SCSD. Two additional schools plsfii to request 
in-service time for workshops. 
F. Junior High Summary 

Chart 8 summarizes the most recently available results for the 
junior high school program segment for students, feachers, parents 
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and business and industry. All student data is based on 1971-72 
end-o£-year data. Mid-year 1972-73 data was no^ available for this 
report. In general, two o£ the student goals were realized for the 
general junior high program (all grades receiving the same material 
'l971-72) and two were not. Career choice awareness and po'Sitive 
attitudes about career planning did not increase. Indications of 
teachers use of materials, parental exposure to GOOP information, 
and business and industry cooperation are positive. 

It is' significant to note th^t, although problems concerning 
curriculum units and teacher acceptance of career education have 
been encountered, GOOP administrators have made program andiiimate- 
rial adjustments and have benefitted from these encounters. Based 
upon incomplete data and observations, all three grade levels of 
the junior high pi:ogram segment appear to be making reasonable 
progress toward stated goals. 
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Chart 7 

Junior High School Teacher Assessment and Use of GOOP Materials 

Percentages 



Definitely 
Yes 



Definitely 
No 



ERIC 



1 2 3 4.5 


Total 

i\v7 O L> WHO w o 

84 


1. Do you have easy 

access to resource , 

material kits? 67 21 ' 6 2 3 


Z. i r you -are not now 

•trsdng the kits, do you 
plan to use them in ^ 

the near future? 50 18 19 2 1 


1 

62 


3. ^ Do the kits provide \ 
an aid to the teach- 
ing o£ career edu- 
cation with^in your 
regular subject 

framcy^ptrkr ZZ Zo Zb lu 


! 
1 

U U ft 


4, D6 you.i^ind the kits 
help£ui\ in teaching 

y U Cl 1 1 LXJLdl OLiUJCV,<U 

matejial? 14 .19 19 29 . 18 


72 


5. Do you feel you are 

well enough acquainted 
with kits to make 
sufficient use of 

them? 17 22 29 20 11 


81 


Veryv Not 
ReRegulajrly , At 

All 




6. To what degree do / . 
you find yourself 

using the kits? 7 15- 24 21 ^0 


78 
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Chart 8 

Suimnary of Junior High Program 



Variable* / Desired Effect Actual Effect 

"By GOOP ^ (Based on most 

current data) 



i. 


Students C7th; 8th, 
and 9j.th") j 




• 




A. Career choice 
awareness 


Increase 


7th Slight Increase 
8th Slight Increase 
9th'Sybstantial 
^ . Increase ' 




B. T/pes of Vocational 
Information Sources 

• > 


Seek more qual- 
if ied jsources 


,* Less Dependency upon 
friends and r^elatives 
and more use of 
counselors and 
employees • 




C, Level of career 
awareness. 


Increase 


Marked Increase 




D. Continue attitudes 
about career plan-;* 
ning 


Increase 


7th Slight Decrease 
8th Slight iDncrease 
9th Slight Increase 


II. 


Teachers 








Teacher use of career 
materials 


Increase 


Requests for GOOP 
help' and. assistance 
have increased 


nr. 


Parents 








Parental exposure to 
junior high program 


Increase 


Output of messages 
about GOOP ^Jias been 
significant 


IV. 


Business and Industry 








Openness to student 
inquiry 


High 


Response to date , 
mainly favorable 
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Chapter V ^, ' 

Evaluation of Senior High School Program 



The Senior High School program segment o£ the Guided Occupational 
Orientation Training Project i^s directed toward the goal o£ job train- 
in^ and placement "Enable individualized career preparation and 
implementation o£ career plan", 

To accomplish this goal, two complimentary program ^ thrus ts are 
employed. First, drop-outs, potential drop-outs and under achievers 
are considered a target population for 'individualized counselling, 
special instruction, and job placement where emgl^i^iiient is desired. 



Second, all senior high school students a*re a target population 
for increasing awareness., for career opportunities .v The Occupational 
Resource Specialists (ORS) described in Chapter IV of this report 
produce and publish a bi-monthly newsletter which is available to 
all senior high school students. The ORS team also assists senior, 
high school guidance counselors to interpret results of the Ohio 
Vocational Interest Survey. 

These activities ire directed toward 5^470 senior high school 
students. ^ 

The target population of drop-outs, potential drop-outs, an^l 
under-achievers is' being served through three program efforts. They 
are as follows . * ' 

(46)' . ' 
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.A. Occupational Learning Center (OLCJ' - Senior high scho.ol 

drop-outs and severe discijpline problem students axe " • . 
served in specialized centers away from actual schools 
This program provides an individualized^ approach that is 
occupationally-oriented rather than academic\ This / 
approach is -combined with actual work experience and/or 
vocational training. , In the 1971-72 school year, 82- 
students were served with oAly ^nine leavihggthe prog-ram. 
At the "liate of this report, 104 students ^re enrolled/ , * . 

B. ' Occupa1;ional Resource Teachers (ORT) -- This program 

*■ ' . . ^ '■ .• > ■ ' 

phase also serves drop-outs, potential drop-outs and% v 
^ • ^ . ^ • Th- 

under-achievers . However, students -are ke^t in the main- 
stream of the school but giyen work experiei;ce opportunities- 
identified by ORT staff mertfbers. , In tjie 1971- 72' school * 
y^ar, 554 students were enrolled in this program phase. 
HoweVef, 177 did not complete activities. 

C. Program for the Advancement .of Career Education . (PACE) . 
The Syracuse City School ^'D^strict and Onondaga- Community 
College a-re cooperating ' to offer PACE. Since its incep- • 
tioii,' the PACE program has emphasized distinctive features 
in furthering the goals of career awareness and skill level 
attainment among high school students-^ ^ Means of achieving 
these goals are through an inter.di3Ciplinary approach that 
includes classroom projects ^n^l related field trips. Through 
PACE, under-achieving, senior high school students are' 
motivated .to remain in school through graduation;* and then 
participate in an associate degree pr-ogram at Onondaga 

(.47) 
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Comftiunity College; Apjjr.oximately 375 students currently 
are participating in PACE, . , ^ • ' • 

Correspanding to design (t£ the Senior High prog^^*am, the 
evaluation will deal separately ^witfi f^e three projects '^^^^ ' 
PACE : ' ' . _ ' ^ ' 

PACE is a multi- facted project including ^our job^families : 
paramedical, engineering technology, retail business management, 
and executive . secretabrial . • ' ^ 

The initial group o£ PACE ' s tudents are now seniors' and will bp 
gradaating in June, -1973. «, 

As an initial step to evaluate PACE, ESI worked with PACE ^taff. 
to develop tentative goals and indicators • A year-end evaluation 
will be complete in June, 1973, . 
A,: Goals and Indicators for PACE * . 

1. "GoaT ^ fifiry percMit-of PACE stt^dlsnt^s- up'dn 'gradiia^tion 
will enter -a one or two year associate degree program.' 

a. . Indicators- entrance pe^^centage at June, 1973 

number accepted as of Deeember, 1972; 

b. PACE counselors will obtain d^ta. 

• ^ 2. Goal - a high percentage of PACE students will i'mprove 
achievement levels during school year. PACE student? 
will demonstrate incr^eases in motivation by remai](^ing m 
school, and pursuing :educationv • ' • 

a. Indicators ' . * " 



(1) - grade point averages June, 1972 ta June , 1973. 
Sample o£ ^1971-72 school- year, 
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^ (2) Drop-out rate or. school attendance , 1971-72- . 

' -school year vs.. 197^^.73 school^year. 
' - (3) Responses to questionnaire- statement, e.gl^, 
. "How tib you -feel aboyt.sctiool?", . and "iHow 
. ' much relation. did yoUr school--woi;k see^gai' to 
J , have on your future?" 

Goal - Increase PACE s'^tudent awareness, particularly con- 

^ corning job clusters, . ^ . . . - 

r ^ ' > ■ ..' ' 

a. Indicators'* ' . Tv . • 

(1) Student assessment of PACE program at end of ^ 

school year Will be favorable^ (on the averag^e) 
' ' \ 

. (.^) PACE students' seljN;:onf idence will inc'rea3e 

comparing entrance measure|s*with end ' of school 

\/ ' 

year measures. (Ins trumeg>ts to be developed) 
(3) S'tudent ^sporrses to questionnaire statements, 



e.g..' "On the whole .how do the students whi 



'■r 



kndtA^ abotrt it seem ^t6 feel about • the PACE* 
program?", "How do your parents feel 'aboi|% tn| 
PACE^rogram? ," "How much confidence do yQU have 
.-in yourself?" and "How good. a student do yop ; 
; try to be?". ' . ' 

* 

Occupational Resource Teachers ' ■ ' 

Only end o£ school data from 1971-72 was available for the 
Occupational .Resource Teachers (ORT) at the time of this report. 
Formal goals for ^hi^' program have not been set at this time, but 
behavoriaX ateas to' be affected- jjy ORT staff activities and results 
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are described b^Low. All data described below was provided by 
ORT staff members, ESI was not involved in specifying data to 
be collected or able to assess data reliability. 

'School attendance for the target group served by ORT staff • 
members consistently belpw average, believed by staff 'members to. 
be less than , thirty percent. Administrators and teacher^s report 
that students ill this program improved in attendance during the 
y©a,r. 'No 'data .on attendances 'was- compiled. 

When asked ,t0^ assess the program, eighty six percent of the' 
students sai'd it was "meaningful", four percen-t said*' it was "not ^ . 
' meaningful", and ten percent did not respond: ' ' 

^ Employers rated their student employees from the program as 
'thi^'ty seven percent excellent, forty six percent satisfactory,' 
arid sevente'en percent needing improvement. " ^ ^ 

Of the 554 :^tudents enrolled in the program, 377 ^studer^ts 
obtained jobs and continued with them throughout thfe. program. 
Occupational Le,;arning Centers' , ' \ 

A. Goals for Students ^ * ' . 

1. Eacii student will increase his basic ski41 competency 
; ' by^ one grade level in re-ading and' math. 

i 

a: Variable - reading, and math gr^de ''levels . 

f 

^. . . ; \ b. Indicator - metropolitan ^standardized test scores. 

Entry and end of school year aVerage grade levels for students 
are ^hown. in Table 16, for vocabulary, reading comprehension; and 
computation (math). Both vocabulary and reading gained aver one 
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grade level -'(1-13 and 1.43 level gains respectively> ^rid matJi - 
gained .91 grade level.. Jhe Average enrollme.nt length was 7.3 
months. Staff members believed grade-level gain would be higher' . 
with larger average enrollnient . The 1972-73 pre-test' scores for 
reading and math are also shown. 



, . ' Table 16 , - ' 

OLC Metropolitan Standardized Test (Grade-Levels) 
' ' 1971-72 



Pre-Test 
at Ent.ry 



Post 
Test 



Grade Level 
Gain 



1972-73 
Pre- te-s t 



Vocabulary 


6.85 


7.98 


1.13 


Not avail. 1 


Reading Comprehension 


6.52 


7.95 


' 1.43 


6.46 


*l]omputation-math 


' 7.03 


7.94 


.91 


■ 6.43 ^ i 



2. ' Students will have twenty five percent improvement .in 
. attendance . ^: , 

a. Variable - level of attendance. 

b. Attendance-i*epords in school and work before entering 
and during program. ^ 

Ave^rage ^attendance at the centers was -82.7 percent as compared 
to 65.4 percent before entering the OLC program, a 17.3 percent 
point gain Ce'^cceeding desired' gain) . Comparing attendance with 
previous attendance, 38 students improved, 34 remained the same, 

and two worsened. \ ' ' * 

■ ^ , \U ■ " '■ 

For 1972-^3, pre-c^nter average attendance was 44.7 percent. 
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,3.-' Each student will find ^significantly measurable success 
'in employment or employment-related straining . 
a. Variabl^e - level of employment or related training 
success . 

;b. Indicator - employer and counselor ratings. 
Sixty-two students had jobs, for the 1971-72 school year with. 
56 students maintaining employment continuously, six having state 
. employm*ent , aud eight not able to or ready to hold a job during the 
year. y For , 1972 \7 3 ,50 students are employed, ^ 20 are, in training , 
and 34^are in vocational readiness. ' . * 

• . - ■ .s • . . ■ 

, Other indicators of success for 1971-72 are shown in Chart 9. ^ 
Other OLC goals for 1972-73 whi^h rela€^ to the end of the 
school year are as follows^ C • 



Students will gain 'in level of general information as indi- 
cated by individualized progress records. 

Student level of career awareness ^will increase as, 

indicated by activity achievement records . 

Students will be able to develop and implement career. 

plans asf indicated by percentage completing career plans 
^. and' vocational dheck list. 



T 
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. ■ ^ . Chart 9 
Employment and Related Training Success Indicators 



1. 

* 


Supervisor's report on job performance - 

* * 

Excellent - 24 students 
Satisfactory - 18 students 
. Needs Improvement - 17 students 






* 




2 . 


Work attendance - percent in relation to possible: 


n 






90-1001 " 59 students 
, 8 0-89 1' - 12 students 

/u /y -5 •) stuuents 
'60-69 % - '4 students ^ ^ • „ 

"^n-QQ ^ " students 




> 






3 • 


V%'ca t ional ReadinessI 


u 


S 


* 

E • 






yy. Positive attitude toward work 

i s ■■ ■ ■ ; i ■ — 


13 


40 


10 


It 


- 


R AHi1i1"v t n irrpnt" tp'^ n ntm i h i 1 i tv 


15 


3 '5 


12 




C. Willingness to do an acceptable job 


10 


41 


21 




D,, Sufficient bccupationq.1 information 


2.0 , 


'37 


6 




£• Choice of career area 


27 


25 ' 


10 


4. 


Preparation for a career ; 










« 


A'. Training of in-ci'epth study in career 
choice 


27 


'30 


5 






' B. Acceptable j ob seeking skills 


16 


26 


8 _ 




C. Employer's evaluation 


9 


24 ■ 


15 


5. 


Work Experience ^ 












A. Sheltered job (lO-JLS hrs. a week) 


9 


21 


11 






'B. .Community job (20-40 hrs. a' wk. paid 
;i by employer) 


9 


20 


12 




C' Obtaining full-time job independently 


4 


15 


4 




D. Plan f0r\[o'st. graduate activity 


1 


6 


2 . 



* U-Unsatis£actory, ^-Satisfactory, - E-Excellent 
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^ Chart 10 
" Sunlmar5^ of Senior Hj.gh Programs 



Variable - Desired Effect ' " Actual Result 
' By GOOP 



I. 


Occupational Resource - 
Teachers ^ 

A. Attendance 


Improve 

• * 






No data, but 
teacher assess- 
ment was "im- • 
provement" 


• 


B. Student assessment 


Favorable 


86^ said program 
was "meaningful" 




C, Employer assessment 


Favorable ^ 


83^ said student 
employees were 
'"satisfactory" or 
•"excellent" 




D, Empldfment 


-Most students 
obtain jobs 


68% obtained and 
maintained jobs 


II. 


Occupational Learning 
Centers 

A, Academic skills 


Improve 


• 




* 

At least one .grade 
level gain 




B Attendance „ ^ ' 


Improve 


• 


17.3% point 
gain 




C. Employment or ^: i 
emp 1 oyme n t re - n 
lated training 


Successfu'l^' 




• 

All indications of 
success were 
• positive 
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Senior High School Summary 

Chart 10 summarizes results for ORT and OLC programs. All 
indicati.ons of student improvement in school and employment- related 
behavior were positive. No evaluation data for PACE was available. 

In March, 1973, ESI evaluation tieam members are scheduled to . 
meet with icey staff members of PACE, OLC, and ORT% This meeting 
will b^e^ designed to assist in development of specific ev,aluation 
techniques. - Also, it will be designed ,to help staff members under- 
stand the stiope and complexity of all senior high school program 
phases operating under GOOP. This lack of understanding ^has i . 

resulted in some fragmentation of program efforts which must be . ' 
corrected. Staff members must become aware of overall program 
goals as well as ' identified goals , for their own program segment. 
' ' Strong efforts must be made by GOOP administrators to 

establish a "team spirit" among the three separate program com- 
ponents. Service, to students in all„ three prdgram segments 
appears to be achieving desired outcomes, but confusion and lack 
of communications are causing ^ ome . undes irable conditions in 
program administrations. Currently, among staff members associated 
with the senior high school program components , there exists a lack , 
• of underatanding of overall program thrusts and goals ; some con- 
fusion concerning program segment relatedness to GOOP' goals; and 
an inability to define'roles of'stz^ff personnel. ^ , 

ESI evaluation team' members have piet separately with, staff 

members of each program segment on several occasion;s. However, 

these individualized meetings have not produced the. broad desired. 

« 

results. i J 

erJc ' :. " •- 



Posfibly, the group meeting in March wpill alleviate problem areas. 
However, GOOP administrators must devote intensive effort to re- 
enforcing this effort and must make every possible effort to 
clarify roles to be assumed by all staff members. 
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Chapter VI - , 
GOOP Administration 

■ ■ . — ■ . 'V 

The administration of the Guided Occupational Orientation Pro- 
ject is-so/nd and is making every effort, to insure that project goals 
are achie)/ed. Administrators have exhibited remarkable flexibility 
in restructuring various program activities in order to achieve 
more progress, • 

As Goop enters the, third year, one grade level program segment 
is exhibiting weakness as related to project administration. The 
three senior high school programj segments — PACE, OLC, and ORT 
represent an administrative problem "centered around authority defin- 
it.ion* A.lthough all three program segments relate to GODP, there 
exist no clear-cut lines of authority to GOOP administrators. This 
situation 'has resulted in confusion and complications which must be 
overcome in the immediate future. ESI evaluation team members re- 
commend that GOOP administrators and staff members of PACE, OLC, and 
ORT "work together to define areas of responsibi and authority. 
GOOP organizational linkage must be developed for the entire senior 
high school program segment. 

The elementary program segment appears well-organized. The fifth 
gtade career awareness program was initiated successfully. The sixth 
grade program segment modifications were introduced effectively and • 
efficiently. ESI team members were impressed with the successful 
efforts of the elementary program staff members in S3|^plifying var- 



ious^testing instruments and in reducing the amount of paper handling 
required for teachers and staff members. 

It is most encouraging to observe the revisions in the junior 
high school pr-ogram segments and materials. These r^evisions were 
made by GOOP administrators in response to ESI evaluation findings. 
Curriculum materials were revamped and revised? specific grade level 
goals were developed by GOOP staff members working with SSI evaluators; 
new curriculum ^materials 'were pilot-testedj and new materials were 
introduced methodically into one school at, a time' rather than on a 
mass basis as in' the past. > 

In the senior high school program segment, the ORT program seg^ 
ment still does not have specific goals established. Goals for the 
PACE program segment are just beginning to emerge but a skeleton set 
of goals was devisei during a brief meeting between PACE staff mem- 
bers and ESI evaluators in December, 1972. ESI staff members re- 
commend that more specific long-term and short-term goals for 'these 
two program segments be developed. A meeting between ESI and senior 
high school program officers is schedul-ed for March, 1973f "to accom- 
plish this. 

Staff members of the Occujational Learning Centers ,iaave estate 
lished^ specific goals, are collecting data in a systematic and ef- 
fective manner, and are responding to evaluation requirements o 

ESI staff members have met with GOOP administrators concetning 
better utilization of the evaluation team's on-sit^e man days ifi 
Syracuse. At the request of ESI staff, a conference with top GOOP 
administrators will be held at the beginning of each on-site visit 
as well as at the conclusion. These conferences will help flag prob- 
lem areas in an orderly manner^ 

(581 



ESI also recommends that the project coordinator be given oppor- 
tunities to visit oirher exemplary projects in the interest of pro- 
fessional growth and -development. This re-commendation as based on 
the provision tl\at travel funds are available within the project 

budget . . , ' . 

Overall, the adjuinistration of GOOP is above average for com- 
parable projects and displays all evidence of dedication to project 
goals and objectives. 
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Chapter VIl ' , ' 

dbservations and Recommendations , • 

The Guided Occupational Orientation Project will enter the third 
and final year of ' federal fundipg on February 1, 1973« This sit- 
uation will create the necessity for administrative decision-making 
throughout the" next 12 months and will create the need for effective 
planning. 

ESI evaluation team members urge that GOOP administrators and 
key staff members begin intensive planning for program continuation 
• after the conclusion of federal funding. Planning should ■ include _ 
attention to co'ntinuation and plans for eventual replacement of ma- 
terials and equipment; disposition of personnel; and the probability - 
of continuing various program ac'tivit'ies . 

In this context, ESI fecommends that an impact study of the 

f . ■ . 

elimination of federal funding of GOOP be completed by project ad- 
ministrator's and staff no later than August, 1, 1973. 

•Chapters V and VI of this report contain, specific recommenda-. 
tions for improving the three program segments concerned with ^enior 
high' school activities. In an overall sense, ESI recommends that • 
GOOP administrators continue to 'Seek improved vmderstanding and co- 
• operation among the program officers of the senior high school com- . 
ponent. As previously stated., it is to be hoped that -work sessions 
scheduled in [VIarch\ 1973 » betVeen GOOP administrators, senior hi^h 
school program officers, ind ESI evaluators will alleviate the sit- 

(6oV * > 
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•uatxon.* 

As the third and final year of. GOOP progressed, ESI proposes 
•to perforin actual program alid it in^' among elementary^ junior high\ ^ 
school, and senior Kigh school tea'chers and students selected rin- 
.4omly. Information gathered during this .-program audit will be in- 
eluded i'h the final* evaluation report. • • ' 

Planning procedures -Have improved ■ greatly since the inception 
of GOOi'. ■ It is significant to more increasing community awareness 
and support, the commitment of Niagra-Mohawk, as evidenced in cor- 
respondence to project administrators J the coverage afforded the 
project. by -Syracuse area mass media; the declaration of the "Year 
of Career Awareness" by the Syracuse Board, of Education; and the 

0 

Ji^itraction of visitors from the community, the state, and the- 
' nation are irtdications of project acceptance. • 

The dissemination of • inf ormat ion concerning GOOP has been ex- . 
cellent. Project staff membters have been' willing to share informa- 
tion and materials'. However, the number of requests for infprmation 
seem to. creie an unnec.essary worK burden for staff members and ; 
must have an adverse effect on administrative costs. ESI recommends 
that some internal evaluation of^ dissemination activities be con- 
ducted promptly." ' '> . 

In assessing the overall impact of GOOP, it is obvious that 
career awareness is increasing generally for students in grades . 
five through ninec However, attitudes are not being affected and" 
it is recommended that attitudinal concepts be discarded in the | 
futures 

In grade ten through twelve, the Occupational Learning Centers 
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are having the desired impact on the target • populations . The work 
activities of the centers are exemplai:y and should be replicated 
in- other school 'districts. It is to be hoped that OLC activities 

» 

can be broaderfed in Syracuse schools in the future, ' . 

ESI recommends that full atte^ntion be focused on exist 3,ng 
program activities in the coming year. . Thia "should "be siijiple to ^ 
accomplish since staff members will be freed from the necessity of 
designing curpriculum materials, developing resource ' lists , and 
introducing faculty members to program activities. The. third E . 
year if GOOP should be the year when ^11 program components operate 
smoothly and efficiently. Staff members should have snore \ime V. 
avai/lable to deal with ^individual problems and concerns. 

Also, ESI urges that members of the Board of Education of the 
Syracuse City School District.- be informed of the impact of C^OOP 
and be urged to make every ef#«rt to continue program actwities 
following the .termination of federal fundipg. 

As in the past, ESI must comment on the dedication of , all 
QflOP staff members the dedication which has overcome^ many 
obstacles and has carried the e^itire project to the/current state 
of general excellence. PartipuLarly , We must remark on the leade.r- 
Ship provided by Mr. Sidney jQhnson, Mr.. Hans Lang, and Mro Richard 
J. Bannigan. " It is impossible to de^tail the many instances when . 
the leadership provided by th^se men has insured the success of 
' the project. . ' . - ■ • 
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